<!-- HTML header for doxygen 1.8.6-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.13"/>
<title>OpenCV: AKAZE and ORB planar tracking</title>
<link href="../../opencv.ico" rel="shortcut icon" type="image/x-icon" />
<link href="../../tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../jquery.js"></script>
<script type="text/javascript" src="../../dynsections.js"></script>
<script type="text/javascript" src="../../tutorial-utils.js"></script>
<link href="../../search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../search/searchdata.js"></script>
<script type="text/javascript" src="../../search/search.js"></script>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
    jax: ["input/TeX","output/HTML-CSS"],
});
//<![CDATA[
MathJax.Hub.Config(
{
  TeX: {
      Macros: {
          matTT: [ "\\[ \\left|\\begin{array}{ccc} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{array}\\right| \\]", 9],
          fork: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ \\end{array} \\right.", 4],
          forkthree: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ \\end{array} \\right.", 6],
          forkfour: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ #7 & \\mbox{#8}\\\\ \\end{array} \\right.", 8],
          vecthree: ["\\begin{bmatrix} #1\\\\ #2\\\\ #3 \\end{bmatrix}", 3],
          vecthreethree: ["\\begin{bmatrix} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{bmatrix}", 9],
          cameramatrix: ["#1 = \\begin{bmatrix} f_x & 0 & c_x\\\\ 0 & f_y & c_y\\\\ 0 & 0 & 1 \\end{bmatrix}", 1],
          distcoeffs: ["(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6 [, s_1, s_2, s_3, s_4[, \\tau_x, \\tau_y]]]]) \\text{ of 4, 5, 8, 12 or 14 elements}"],
          distcoeffsfisheye: ["(k_1, k_2, k_3, k_4)"],
          hdotsfor: ["\\dots", 1],
          mathbbm: ["\\mathbb{#1}", 1],
          bordermatrix: ["\\matrix{#1}", 1]
      }
  }
}
);
//]]>
</script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js"></script>
<link href="../../doxygen.css" rel="stylesheet" type="text/css" />
<link href="../../stylesheet.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<!--#include virtual="/google-search.html"-->
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectlogo"><img alt="Logo" src="../../opencv-logo-small.png"/></td>
  <td style="padding-left: 0.5em;">
   <div id="projectname">OpenCV
   &#160;<span id="projectnumber">4.5.2</span>
   </div>
   <div id="projectbrief">Open Source Computer Vision</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.13 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "../../search",false,'Search');
</script>
<script type="text/javascript" src="../../menudata.js"></script>
<script type="text/javascript" src="../../menu.js"></script>
<script type="text/javascript">
$(function() {
  initMenu('../../',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="../../d9/df8/tutorial_root.html">OpenCV Tutorials</a></li><li class="navelem"><a class="el" href="../../d9/d97/tutorial_table_of_content_features2d.html">2D Features framework (feature2d module)</a></li>  </ul>
</div>
</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">AKAZE and ORB planar tracking </div>  </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p><b>Prev Tutorial:</b> <a class="el" href="../../db/d70/tutorial_akaze_matching.html">AKAZE local features matching</a></p>
<p><b>Next Tutorial:</b> <a class="el" href="../../d9/dab/tutorial_homography.html">Basic concepts of the homography explained with code</a></p>
<table class="doxtable">
<tr>
<th align="right"></th><th align="left"></th></tr>
<tr>
<td align="right">Original author </td><td align="left">Fedor Morozov </td></tr>
<tr>
<td align="right">Compatibility </td><td align="left">OpenCV &gt;= 3.0 </td></tr>
</table>
<h2>Introduction </h2>
<p>In this tutorial we will compare <em>AKAZE</em> and <em>ORB</em> local features using them to find matches between video frames and track object movements.</p>
<p>The algorithm is as follows:</p>
<ul>
<li>Detect and describe keypoints on the first frame, manually set object boundaries</li>
<li>For every next frame:<ol type="1">
<li>Detect and describe keypoints</li>
<li>Match them using bruteforce matcher</li>
<li>Estimate homography transformation using RANSAC</li>
<li>Filter inliers from all the matches</li>
<li>Apply homography transformation to the bounding box to find the object</li>
<li>Draw bounding box and inliers, compute inlier ratio as evaluation metric</li>
</ol>
</li>
</ul>
<div class="image">
<img src="../../frame.png" alt="frame.png"/>
</div>
<h2>Data </h2>
<p>To do the tracking we need a video and object position on the first frame.</p>
<p>You can download our example video and data from <a href="https://docs.google.com/file/d/0B72G7D4snftJandBb0taLVJHMFk">here</a>.</p>
<p>To run the code you have to specify input (camera id or video_file). Then, select a bounding box with the mouse, and press any key to start tracking </p><div class="fragment"><div class="line">./planar_tracking blais.mp4</div></div><!-- fragment --><h2>Source Code </h2>
<div class="fragment"><div class="line"><span class="preprocessor">#include &lt;<a class="code" href="../../d5/d0d/features2d_8hpp.html">opencv2/features2d.hpp</a>&gt;</span></div><div class="line"><span class="preprocessor">#include &lt;<a class="code" href="../../dc/d3d/videoio_8hpp.html">opencv2/videoio.hpp</a>&gt;</span></div><div class="line"><span class="preprocessor">#include &lt;<a class="code" href="../../d1/d4f/imgproc_2include_2opencv2_2imgproc_8hpp.html">opencv2/imgproc.hpp</a>&gt;</span></div><div class="line"><span class="preprocessor">#include &lt;<a class="code" href="../../d2/d28/calib3d_8hpp.html">opencv2/calib3d.hpp</a>&gt;</span></div><div class="line"><span class="preprocessor">#include &lt;<a class="code" href="../../d4/dd5/highgui_8hpp.html">opencv2/highgui.hpp</a>&gt;</span>      <span class="comment">//for imshow</span></div><div class="line"><span class="preprocessor">#include &lt;vector&gt;</span></div><div class="line"><span class="preprocessor">#include &lt;iostream&gt;</span></div><div class="line"><span class="preprocessor">#include &lt;iomanip&gt;</span></div><div class="line"></div><div class="line"><span class="preprocessor">#include &quot;stats.h&quot;</span> <span class="comment">// Stats structure definition</span></div><div class="line"><span class="preprocessor">#include &quot;utils.h&quot;</span> <span class="comment">// Drawing and printing functions</span></div><div class="line"></div><div class="line"><span class="keyword">using namespace </span>std;</div><div class="line"><span class="keyword">using namespace </span><a class="code" href="../../d2/d75/namespacecv.html">cv</a>;</div><div class="line"></div><div class="line"><span class="keyword">const</span> <span class="keywordtype">double</span> akaze_thresh = 3e-4; <span class="comment">// AKAZE detection threshold set to locate about 1000 keypoints</span></div><div class="line"><span class="keyword">const</span> <span class="keywordtype">double</span> ransac_thresh = 2.5f; <span class="comment">// RANSAC inlier threshold</span></div><div class="line"><span class="keyword">const</span> <span class="keywordtype">double</span> nn_match_ratio = 0.8f; <span class="comment">// Nearest-neighbour matching ratio</span></div><div class="line"><span class="keyword">const</span> <span class="keywordtype">int</span> bb_min_inliers = 100; <span class="comment">// Minimal number of inliers to draw bounding box</span></div><div class="line"><span class="keyword">const</span> <span class="keywordtype">int</span> stats_update_period = 10; <span class="comment">// On-screen statistics are updated every 10 frames</span></div><div class="line"></div><div class="line"><span class="keyword">namespace </span>example {</div><div class="line"><span class="keyword">class </span><a class="code" href="../../d0/d0a/classcv_1_1Tracker.html">Tracker</a></div><div class="line">{</div><div class="line"><span class="keyword">public</span>:</div><div class="line">    <a class="code" href="../../d0/d0a/classcv_1_1Tracker.html">Tracker</a>(<a class="code" href="../../dc/d84/group__core__basic.html#ga6395ca871a678020c4a31fadf7e8cc63">Ptr&lt;Feature2D&gt;</a> _detector, <a class="code" href="../../dc/d84/group__core__basic.html#ga6395ca871a678020c4a31fadf7e8cc63">Ptr&lt;DescriptorMatcher&gt;</a> _matcher) :</div><div class="line">        detector(_detector),</div><div class="line">        matcher(_matcher)</div><div class="line">    {}</div><div class="line"></div><div class="line">    <span class="keywordtype">void</span> setFirstFrame(<span class="keyword">const</span> <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> frame, vector&lt;Point2f&gt; bb, <span class="keywordtype">string</span> title, Stats&amp; stats);</div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> process(<span class="keyword">const</span> <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> frame, Stats&amp; stats);</div><div class="line">    <a class="code" href="../../dc/d84/group__core__basic.html#ga6395ca871a678020c4a31fadf7e8cc63">Ptr&lt;Feature2D&gt;</a> getDetector() {</div><div class="line">        <span class="keywordflow">return</span> detector;</div><div class="line">    }</div><div class="line"><span class="keyword">protected</span>:</div><div class="line">    <a class="code" href="../../dc/d84/group__core__basic.html#ga6395ca871a678020c4a31fadf7e8cc63">Ptr&lt;Feature2D&gt;</a> detector;</div><div class="line">    <a class="code" href="../../dc/d84/group__core__basic.html#ga6395ca871a678020c4a31fadf7e8cc63">Ptr&lt;DescriptorMatcher&gt;</a> matcher;</div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> first_frame, first_desc;</div><div class="line">    vector&lt;KeyPoint&gt; first_kp;</div><div class="line">    vector&lt;Point2f&gt; object_bb;</div><div class="line">};</div><div class="line"></div><div class="line"><span class="keywordtype">void</span> Tracker::setFirstFrame(<span class="keyword">const</span> <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> frame, vector&lt;Point2f&gt; bb, <span class="keywordtype">string</span> title, Stats&amp; stats)</div><div class="line">{</div><div class="line">    <a class="code" href="../../db/d4e/classcv_1_1Point__.html">cv::Point</a> *ptMask = <span class="keyword">new</span> <a class="code" href="../../db/d4e/classcv_1_1Point__.html">cv::Point</a>[bb.size()];</div><div class="line">    <span class="keyword">const</span> <a class="code" href="../../db/d4e/classcv_1_1Point__.html">Point</a>* ptContain = { &amp;ptMask[0] };</div><div class="line">    <span class="keywordtype">int</span> iSize = <span class="keyword">static_cast&lt;</span><span class="keywordtype">int</span><span class="keyword">&gt;</span>(bb.size());</div><div class="line">    <span class="keywordflow">for</span> (<span class="keywordtype">size_t</span> i=0; i&lt;bb.size(); i++) {</div><div class="line">        ptMask[i].<a class="code" href="../../db/d4e/classcv_1_1Point__.html#a4c96fa7bdbfe390be5ed356edb274ff3">x</a> = <span class="keyword">static_cast&lt;</span><span class="keywordtype">int</span><span class="keyword">&gt;</span>(bb[i].x);</div><div class="line">        ptMask[i].<a class="code" href="../../db/d4e/classcv_1_1Point__.html#a157337197338ff199e5df1a393022f15">y</a> = <span class="keyword">static_cast&lt;</span><span class="keywordtype">int</span><span class="keyword">&gt;</span>(bb[i].y);</div><div class="line">    }</div><div class="line">    first_frame = frame.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#adff2ea98da45eae0833e73582dd4a660">clone</a>();</div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">cv::Mat</a> matMask = <a class="code" href="../../d3/d63/classcv_1_1Mat.html#a0b57b6a326c8876d944d188a46e0f556">cv::Mat::zeros</a>(frame.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#a146f8e8dda07d1365a575ab83d9828d1">size</a>(), <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga81df635441b21f532fdace401e04f588">CV_8UC1</a>);</div><div class="line">    <a class="code" href="../../d6/d6e/group__imgproc__draw.html#ga8c69b68fab5f25e2223b6496aa60dad5">cv::fillPoly</a>(matMask, &amp;ptContain, &amp;iSize, 1, <a class="code" href="../../d1/da0/classcv_1_1Scalar__.html#ac1509a4b8454fe7fe29db069e13a2e6f">cv::Scalar::all</a>(255));</div><div class="line">    detector-&gt;<a class="code" href="../../d0/d13/classcv_1_1Feature2D.html#a8be0d1c20b08eb867184b8d74c15a677">detectAndCompute</a>(first_frame, matMask, first_kp, first_desc);</div><div class="line">    stats.keypoints = (int)first_kp.size();</div><div class="line">    drawBoundingBox(first_frame, bb);</div><div class="line">    <a class="code" href="../../d6/d6e/group__imgproc__draw.html#ga5126f47f883d730f633d74f07456c576">putText</a>(first_frame, title, <a class="code" href="../../dc/d84/group__core__basic.html#ga1e83eafb2d26b3c93f09e8338bcab192">Point</a>(0, 60), <a class="code" href="../../d6/d6e/group__imgproc__draw.html#gga0f9314ea6e35f99bb23f29567fc16e11a08cf3b0a37729fbb62a3007d499cbd8b">FONT_HERSHEY_PLAIN</a>, 5, Scalar::all(0), 4);</div><div class="line">    object_bb = bb;</div><div class="line">    <span class="keyword">delete</span>[] ptMask;</div><div class="line">}</div><div class="line"></div><div class="line"><a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> Tracker::process(<span class="keyword">const</span> <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> frame, Stats&amp; stats)</div><div class="line">{</div><div class="line">    <a class="code" href="../../d9/d6f/classcv_1_1TickMeter.html">TickMeter</a> tm;</div><div class="line">    vector&lt;KeyPoint&gt; kp;</div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> desc;</div><div class="line"></div><div class="line">    tm.<a class="code" href="../../d9/d6f/classcv_1_1TickMeter.html#a564bc8b0bebcfc6c96f85fc48643e273">start</a>();</div><div class="line">    detector-&gt;<a class="code" href="../../d0/d13/classcv_1_1Feature2D.html#a8be0d1c20b08eb867184b8d74c15a677">detectAndCompute</a>(frame, <a class="code" href="../../dc/d84/group__core__basic.html#gad9287b23bba2fed753b36ef561ae7346">noArray</a>(), kp, desc);</div><div class="line">    stats.keypoints = (int)kp.size();</div><div class="line"></div><div class="line">    vector&lt; vector&lt;DMatch&gt; &gt; matches;</div><div class="line">    vector&lt;KeyPoint&gt; matched1, matched2;</div><div class="line">    matcher-&gt;knnMatch(first_desc, desc, matches, 2);</div><div class="line">    <span class="keywordflow">for</span>(<span class="keywordtype">unsigned</span> i = 0; i &lt; matches.size(); i++) {</div><div class="line">        <span class="keywordflow">if</span>(matches[i][0].distance &lt; nn_match_ratio * matches[i][1].distance) {</div><div class="line">            matched1.push_back(first_kp[matches[i][0].queryIdx]);</div><div class="line">            matched2.push_back(      kp[matches[i][0].trainIdx]);</div><div class="line">        }</div><div class="line">    }</div><div class="line">    stats.matches = (int)matched1.size();</div><div class="line"></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> inlier_mask, homography;</div><div class="line">    vector&lt;KeyPoint&gt; inliers1, inliers2;</div><div class="line">    vector&lt;DMatch&gt; inlier_matches;</div><div class="line">    <span class="keywordflow">if</span>(matched1.size() &gt;= 4) {</div><div class="line">        homography = <a class="code" href="../../d9/d0c/group__calib3d.html#ga4abc2ece9fab9398f2e560d53c8c9780">findHomography</a>(Points(matched1), Points(matched2),</div><div class="line">                                    <a class="code" href="../../d9/d0c/group__calib3d.html#gga58a3bc75c7337534ea9c1697b928cae0a724159df258a5d7e29410a6a2f4e6c87">RANSAC</a>, ransac_thresh, inlier_mask);</div><div class="line">    }</div><div class="line">    tm.<a class="code" href="../../d9/d6f/classcv_1_1TickMeter.html#aa6dfbdc68373d10e371bd35f92c7adb3">stop</a>();</div><div class="line">    stats.fps = 1. / tm.<a class="code" href="../../d9/d6f/classcv_1_1TickMeter.html#a9e21f1bdbe5089c7288b8df81e5ac4a1">getTimeSec</a>();</div><div class="line"></div><div class="line">    <span class="keywordflow">if</span>(matched1.size() &lt; 4 || homography.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#abbec3525a852e77998aba034813fded4">empty</a>()) {</div><div class="line">        <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> res;</div><div class="line">        <a class="code" href="../../d2/de8/group__core__array.html#gaf9771c991763233866bf76b5b5d1776f">hconcat</a>(first_frame, frame, res);</div><div class="line">        stats.inliers = 0;</div><div class="line">        stats.ratio = 0;</div><div class="line">        <span class="keywordflow">return</span> res;</div><div class="line">    }</div><div class="line">    <span class="keywordflow">for</span>(<span class="keywordtype">unsigned</span> i = 0; i &lt; matched1.size(); i++) {</div><div class="line">        <span class="keywordflow">if</span>(inlier_mask.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aa5d20fc86d41d59e4d71ae93daee9726">at</a>&lt;<a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga65f85814a8290f9797005d3b28e7e5fc">uchar</a>&gt;(i)) {</div><div class="line">            <span class="keywordtype">int</span> new_i = <span class="keyword">static_cast&lt;</span><span class="keywordtype">int</span><span class="keyword">&gt;</span>(inliers1.size());</div><div class="line">            inliers1.push_back(matched1[i]);</div><div class="line">            inliers2.push_back(matched2[i]);</div><div class="line">            inlier_matches.push_back(<a class="code" href="../../d4/de0/classcv_1_1DMatch.html">DMatch</a>(new_i, new_i, 0));</div><div class="line">        }</div><div class="line">    }</div><div class="line">    stats.inliers = (int)inliers1.size();</div><div class="line">    stats.ratio = stats.inliers * 1.0 / stats.matches;</div><div class="line"></div><div class="line">    vector&lt;Point2f&gt; new_bb;</div><div class="line">    <a class="code" href="../../d2/de8/group__core__array.html#gad327659ac03e5fd6894b90025e6900a7">perspectiveTransform</a>(object_bb, new_bb, homography);</div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> frame_with_bb = frame.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#adff2ea98da45eae0833e73582dd4a660">clone</a>();</div><div class="line">    <span class="keywordflow">if</span>(stats.inliers &gt;= bb_min_inliers) {</div><div class="line">        drawBoundingBox(frame_with_bb, new_bb);</div><div class="line">    }</div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> res;</div><div class="line">    <a class="code" href="../../d4/d5d/group__features2d__draw.html#gad8f463ccaf0dc6f61083abd8717c261a">drawMatches</a>(first_frame, inliers1, frame_with_bb, inliers2,</div><div class="line">                inlier_matches, res,</div><div class="line">                <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(255, 0, 0), <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(255, 0, 0));</div><div class="line">    <span class="keywordflow">return</span> res;</div><div class="line">}</div><div class="line">}</div><div class="line"></div><div class="line"><span class="keywordtype">int</span> main(<span class="keywordtype">int</span> argc, <span class="keywordtype">char</span> **argv)</div><div class="line">{</div><div class="line">    <a class="code" href="../../d0/d2e/classcv_1_1CommandLineParser.html">CommandLineParser</a> parser(argc, argv, <span class="stringliteral">&quot;{@input_path |0|input path can be a camera id, like 0,1,2 or a video filename}&quot;</span>);</div><div class="line">    parser.printMessage();</div><div class="line">    <span class="keywordtype">string</span> input_path = parser.get&lt;<span class="keywordtype">string</span>&gt;(0);</div><div class="line">    <span class="keywordtype">string</span> video_name = input_path;</div><div class="line"></div><div class="line">    <a class="code" href="../../d8/dfe/classcv_1_1VideoCapture.html">VideoCapture</a> video_in;</div><div class="line"></div><div class="line">    <span class="keywordflow">if</span> ( ( isdigit(input_path[0]) &amp;&amp; input_path.size() == 1 ) )</div><div class="line">    {</div><div class="line">    <span class="keywordtype">int</span> camera_no = input_path[0] - <span class="charliteral">&#39;0&#39;</span>;</div><div class="line">        video_in.<a class="code" href="../../d8/dfe/classcv_1_1VideoCapture.html#a614a1702e15f42ede5100014ce7f48ed">open</a>( camera_no );</div><div class="line">    }</div><div class="line">    <span class="keywordflow">else</span> {</div><div class="line">        video_in.<a class="code" href="../../d8/dfe/classcv_1_1VideoCapture.html#a614a1702e15f42ede5100014ce7f48ed">open</a>(video_name);</div><div class="line">    }</div><div class="line"></div><div class="line">    <span class="keywordflow">if</span>(!video_in.<a class="code" href="../../d8/dfe/classcv_1_1VideoCapture.html#a9d2ca36789e7fcfe7a7be3b328038585">isOpened</a>()) {</div><div class="line">        cerr &lt;&lt; <span class="stringliteral">&quot;Couldn&#39;t open &quot;</span> &lt;&lt; video_name &lt;&lt; endl;</div><div class="line">        <span class="keywordflow">return</span> 1;</div><div class="line">    }</div><div class="line"></div><div class="line">    Stats stats, akaze_stats, orb_stats;</div><div class="line">    <a class="code" href="../../dc/d84/group__core__basic.html#ga6395ca871a678020c4a31fadf7e8cc63">Ptr&lt;AKAZE&gt;</a> akaze = AKAZE::create();</div><div class="line">    akaze-&gt;setThreshold(akaze_thresh);</div><div class="line">    <a class="code" href="../../dc/d84/group__core__basic.html#ga6395ca871a678020c4a31fadf7e8cc63">Ptr&lt;ORB&gt;</a> orb = ORB::create();</div><div class="line">    <a class="code" href="../../dc/d84/group__core__basic.html#ga6395ca871a678020c4a31fadf7e8cc63">Ptr&lt;DescriptorMatcher&gt;</a> matcher = DescriptorMatcher::create(<span class="stringliteral">&quot;BruteForce-Hamming&quot;</span>);</div><div class="line">    example::Tracker akaze_tracker(akaze, matcher);</div><div class="line">    example::Tracker orb_tracker(orb, matcher);</div><div class="line"></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> frame;</div><div class="line">    <a class="code" href="../../d7/dfc/group__highgui.html#ga5afdf8410934fd099df85c75b2e0888b">namedWindow</a>(video_name, <a class="code" href="../../d0/d90/group__highgui__window__flags.html#ggabf7d2c5625bc59ac130287f925557ac3a29e45c5af696f73ce5e153601e5ca0f1">WINDOW_NORMAL</a>);</div><div class="line">    cout &lt;&lt; <span class="stringliteral">&quot;\nPress any key to stop the video and select a bounding box&quot;</span> &lt;&lt; endl;</div><div class="line"></div><div class="line">    <span class="keywordflow">while</span> ( <a class="code" href="../../d7/dfc/group__highgui.html#ga5628525ad33f52eab17feebcfba38bd7">waitKey</a>(1) &lt; 1 )</div><div class="line">    {</div><div class="line">        video_in &gt;&gt; frame;</div><div class="line">        <a class="code" href="../../d7/dfc/group__highgui.html#ga9e80e080f7ef33f897e415358aee7f7e">cv::resizeWindow</a>(video_name, frame.size());</div><div class="line">        <a class="code" href="../../d7/dfc/group__highgui.html#ga453d42fe4cb60e5723281a89973ee563">imshow</a>(video_name, frame);</div><div class="line">    }</div><div class="line"></div><div class="line">    vector&lt;Point2f&gt; bb;</div><div class="line">    <a class="code" href="../../d2/d44/classcv_1_1Rect__.html">cv::Rect</a> uBox = <a class="code" href="../../d7/dfc/group__highgui.html#ga8daf4730d3adf7035b6de9be4c469af5">cv::selectROI</a>(video_name, frame);</div><div class="line">    bb.push_back(<a class="code" href="../../db/d4e/classcv_1_1Point__.html">cv::Point2f</a>(static_cast&lt;float&gt;(uBox.<a class="code" href="../../d2/d44/classcv_1_1Rect__.html#a2cadfdc3b4b7dbf8085622b27e044572">x</a>), static_cast&lt;float&gt;(uBox.<a class="code" href="../../d2/d44/classcv_1_1Rect__.html#a6a4860e984df1752623b6ce2a8bde73a">y</a>)));</div><div class="line">    bb.push_back(<a class="code" href="../../db/d4e/classcv_1_1Point__.html">cv::Point2f</a>(static_cast&lt;float&gt;(uBox.<a class="code" href="../../d2/d44/classcv_1_1Rect__.html#a2cadfdc3b4b7dbf8085622b27e044572">x</a>+uBox.<a class="code" href="../../d2/d44/classcv_1_1Rect__.html#a6c16a3bce912faa4fe5be42d7f1b53fe">width</a>), static_cast&lt;float&gt;(uBox.<a class="code" href="../../d2/d44/classcv_1_1Rect__.html#a6a4860e984df1752623b6ce2a8bde73a">y</a>)));</div><div class="line">    bb.push_back(<a class="code" href="../../db/d4e/classcv_1_1Point__.html">cv::Point2f</a>(static_cast&lt;float&gt;(uBox.<a class="code" href="../../d2/d44/classcv_1_1Rect__.html#a2cadfdc3b4b7dbf8085622b27e044572">x</a>+uBox.<a class="code" href="../../d2/d44/classcv_1_1Rect__.html#a6c16a3bce912faa4fe5be42d7f1b53fe">width</a>), static_cast&lt;float&gt;(uBox.<a class="code" href="../../d2/d44/classcv_1_1Rect__.html#a6a4860e984df1752623b6ce2a8bde73a">y</a>+uBox.<a class="code" href="../../d2/d44/classcv_1_1Rect__.html#a6fed06513cedd76652389e38c7b1222e">height</a>)));</div><div class="line">    bb.push_back(<a class="code" href="../../db/d4e/classcv_1_1Point__.html">cv::Point2f</a>(static_cast&lt;float&gt;(uBox.<a class="code" href="../../d2/d44/classcv_1_1Rect__.html#a2cadfdc3b4b7dbf8085622b27e044572">x</a>), static_cast&lt;float&gt;(uBox.<a class="code" href="../../d2/d44/classcv_1_1Rect__.html#a6a4860e984df1752623b6ce2a8bde73a">y</a>+uBox.<a class="code" href="../../d2/d44/classcv_1_1Rect__.html#a6fed06513cedd76652389e38c7b1222e">height</a>)));</div><div class="line"></div><div class="line">    akaze_tracker.setFirstFrame(frame, bb, <span class="stringliteral">&quot;AKAZE&quot;</span>, stats);</div><div class="line">    orb_tracker.setFirstFrame(frame, bb, <span class="stringliteral">&quot;ORB&quot;</span>, stats);</div><div class="line"></div><div class="line">    Stats akaze_draw_stats, orb_draw_stats;</div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> akaze_res, orb_res, res_frame;</div><div class="line">    <span class="keywordtype">int</span> i = 0;</div><div class="line">    <span class="keywordflow">for</span>(;;) {</div><div class="line">        i++;</div><div class="line">        <span class="keywordtype">bool</span> update_stats = (i % stats_update_period == 0);</div><div class="line">        video_in &gt;&gt; frame;</div><div class="line">        <span class="comment">// stop the program if no more images</span></div><div class="line">        <span class="keywordflow">if</span>(frame.empty()) <span class="keywordflow">break</span>;</div><div class="line"></div><div class="line">        akaze_res = akaze_tracker.process(frame, stats);</div><div class="line">        akaze_stats += stats;</div><div class="line">        <span class="keywordflow">if</span>(update_stats) {</div><div class="line">            akaze_draw_stats = stats;</div><div class="line">        }</div><div class="line"></div><div class="line">        orb-&gt;setMaxFeatures(stats.keypoints);</div><div class="line">        orb_res = orb_tracker.process(frame, stats);</div><div class="line">        orb_stats += stats;</div><div class="line">        <span class="keywordflow">if</span>(update_stats) {</div><div class="line">            orb_draw_stats = stats;</div><div class="line">        }</div><div class="line"></div><div class="line">        drawStatistics(akaze_res, akaze_draw_stats);</div><div class="line">        drawStatistics(orb_res, orb_draw_stats);</div><div class="line">        <a class="code" href="../../d2/de8/group__core__array.html#ga744f53b69f6e4f12156cdde4e76aed27">vconcat</a>(akaze_res, orb_res, res_frame);</div><div class="line">        <a class="code" href="../../d7/dfc/group__highgui.html#ga453d42fe4cb60e5723281a89973ee563">cv::imshow</a>(video_name, res_frame);</div><div class="line">        <span class="keywordflow">if</span>(<a class="code" href="../../d7/dfc/group__highgui.html#ga5628525ad33f52eab17feebcfba38bd7">waitKey</a>(1)==27) <span class="keywordflow">break</span>; <span class="comment">//quit on ESC button</span></div><div class="line">    }</div><div class="line">    akaze_stats /= i - 1;</div><div class="line">    orb_stats /= i - 1;</div><div class="line">    printStatistics(<span class="stringliteral">&quot;AKAZE&quot;</span>, akaze_stats);</div><div class="line">    printStatistics(<span class="stringliteral">&quot;ORB&quot;</span>, orb_stats);</div><div class="line">    <span class="keywordflow">return</span> 0;</div><div class="line">}</div></div><!-- fragment --><h2>Explanation </h2>
<h3>Tracker class</h3>
<p>This class implements algorithm described abobve using given feature detector and descriptor matcher.</p>
<ul>
<li><p class="startli"><b>Setting up the first frame</b> </p><div class="fragment"><div class="line"><span class="keywordtype">void</span> Tracker::setFirstFrame(<span class="keyword">const</span> Mat frame, vector&lt;Point2f&gt; bb, <span class="keywordtype">string</span> title, Stats&amp; stats)</div><div class="line">{</div><div class="line">    first_frame = frame.clone();</div><div class="line">    (*detector)(first_frame, <a class="code" href="../../dc/d84/group__core__basic.html#gad9287b23bba2fed753b36ef561ae7346">noArray</a>(), first_kp, first_desc);</div><div class="line">    stats.keypoints = (int)first_kp.size();</div><div class="line">    drawBoundingBox(first_frame, bb);</div><div class="line">    <a class="code" href="../../d6/d6e/group__imgproc__draw.html#ga5126f47f883d730f633d74f07456c576">putText</a>(first_frame, title, <a class="code" href="../../dc/d84/group__core__basic.html#ga1e83eafb2d26b3c93f09e8338bcab192">Point</a>(0, 60), <a class="code" href="../../d6/d6e/group__imgproc__draw.html#gga0f9314ea6e35f99bb23f29567fc16e11a08cf3b0a37729fbb62a3007d499cbd8b">FONT_HERSHEY_PLAIN</a>, 5, Scalar::all(0), 4);</div><div class="line">    object_bb = bb;</div><div class="line">}</div></div><!-- fragment --><p> We compute and store keypoints and descriptors from the first frame and prepare it for the output.</p>
<p class="startli">We need to save number of detected keypoints to make sure both detectors locate roughly the same number of those.</p>
</li>
<li><b>Processing frames</b><ol type="1">
<li><p class="startli">Locate keypoints and compute descriptors </p><div class="fragment"><div class="line">(*detector)(frame, <a class="code" href="../../dc/d84/group__core__basic.html#gad9287b23bba2fed753b36ef561ae7346">noArray</a>(), kp, desc);</div></div><!-- fragment --><p class="startli">To find matches between frames we have to locate the keypoints first.</p>
<p class="startli">In this tutorial detectors are set up to find about 1000 keypoints on each frame.</p>
</li>
<li>Use 2-nn matcher to find correspondences <div class="fragment"><div class="line">matcher-&gt;knnMatch(first_desc, desc, matches, 2);</div><div class="line"><span class="keywordflow">for</span>(<span class="keywordtype">unsigned</span> i = 0; i &lt; matches.size(); i++) {</div><div class="line">    <span class="keywordflow">if</span>(matches[i][0].distance &lt; nn_match_ratio * matches[i][1].distance) {</div><div class="line">        matched1.push_back(first_kp[matches[i][0].queryIdx]);</div><div class="line">        matched2.push_back(      kp[matches[i][0].trainIdx]);</div><div class="line">    }</div><div class="line">}</div></div><!-- fragment --> If the closest match is <em>nn_match_ratio</em> closer than the second closest one, then it's a match.</li>
<li>Use <em>RANSAC</em> to estimate homography transformation <div class="fragment"><div class="line">homography = <a class="code" href="../../d9/d0c/group__calib3d.html#ga4abc2ece9fab9398f2e560d53c8c9780">findHomography</a>(Points(matched1), Points(matched2),</div><div class="line">                            <a class="code" href="../../d9/d0c/group__calib3d.html#gga58a3bc75c7337534ea9c1697b928cae0a724159df258a5d7e29410a6a2f4e6c87">RANSAC</a>, ransac_thresh, inlier_mask);</div></div><!-- fragment --> If there are at least 4 matches we can use random sample consensus to estimate image transformation.</li>
<li>Save the inliers <div class="fragment"><div class="line"><span class="keywordflow">for</span>(<span class="keywordtype">unsigned</span> i = 0; i &lt; matched1.size(); i++) {</div><div class="line">    <span class="keywordflow">if</span>(inlier_mask.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#aa5d20fc86d41d59e4d71ae93daee9726">at</a>&lt;<a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga65f85814a8290f9797005d3b28e7e5fc">uchar</a>&gt;(i)) {</div><div class="line">        <span class="keywordtype">int</span> new_i = <span class="keyword">static_cast&lt;</span><span class="keywordtype">int</span><span class="keyword">&gt;</span>(inliers1.size());</div><div class="line">        inliers1.push_back(matched1[i]);</div><div class="line">        inliers2.push_back(matched2[i]);</div><div class="line">        inlier_matches.push_back(DMatch(new_i, new_i, 0));</div><div class="line">    }</div><div class="line">}</div></div><!-- fragment --> Since <em>findHomography</em> computes the inliers we only have to save the chosen points and matches.</li>
<li><p class="startli">Project object bounding box </p><div class="fragment"><div class="line"><a class="code" href="../../d2/de8/group__core__array.html#gad327659ac03e5fd6894b90025e6900a7">perspectiveTransform</a>(object_bb, new_bb, homography);</div></div><!-- fragment --><p class="startli">If there is a reasonable number of inliers we can use estimated transformation to locate the object.</p>
</li>
</ol>
</li>
</ul>
<h2>Results </h2>
<p>You can watch the resulting <a href="http://www.youtube.com/watch?v=LWY-w8AGGhE">video on youtube</a>.</p>
<p><em>AKAZE</em> statistics: </p><div class="fragment"><div class="line">Matches      626</div><div class="line">Inliers      410</div><div class="line">Inlier ratio 0.58</div><div class="line">Keypoints    1117</div></div><!-- fragment --><p><em>ORB</em> statistics: </p><div class="fragment"><div class="line">Matches      504</div><div class="line">Inliers      319</div><div class="line">Inlier ratio 0.56</div><div class="line">Keypoints    1112</div></div><!-- fragment --> </div></div><!-- contents -->
<!-- HTML footer for doxygen 1.8.6-->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Fri Apr 2 2021 11:36:34 for OpenCV by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="../../doxygen.png" alt="doxygen"/>
</a> 1.8.13
</small></address>
<script type="text/javascript">
//<![CDATA[
addTutorialsButtons();
//]]>
</script>
</body>
</html>
